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Techniques of Assessing Mental Effort 



Although the fact that learners invest less mental effort in television instruction than in 
print-based instruction has been documented (Salomon, 1983; Salomon, 1984; Salomon & 
Leigh, 1984), the search for techniques to increase the effort that leamers invest in video- 
based instruction has been hindered by the limitations of the instruments used to assess the 
construct of mental effort. 

Early work (Salomon, 1983; Salomon, 1984; Salomon & Leigh, 1984) that investigated 
the amount of effort leamers invested in print and video-based instruction used self-report 
questionnaires to documenl the learners' effort expenditures. Recently, however, researchers 
(Beentjes, 1989; Cennamo, Savenye, & Smith, 1991) have identified a need for more precise 
methods of assessing mental effort. 

A recent study (Cennamo, et. al., 1991) incorporated a questioning technique that has 
been shown to increase the effort invested in print-based materials (Britton, Piha, Davis, & 
Wehausen E. 1978; Burton, Niles, & Lalik,1986) into video-based materials to determine 
whether the technique effectively increased the effort invested in video and print in a similar 
manner. Although there were significant differences among the groups on a test of cued recall, 
there were no significant differences among the groups in the amount of mental effort reported 
on a self-report questionnaire. It is the opinion of the researchers and several anonymous 
reviewers of the manuscript that the findings of the study were severely limited by the 
instrument used to assess mental effort. Mental effort was assessed by means of a questionnaire 
that was developed by combining the questions used by Salomon in two studies (1983; 1984). 
Although Salomon reported an acceptable level of reliability in his studies, in this study, the 
Cronbach's alpha of .55 was unacceptably low. 

Beentjes (1989) attempted to replicate Salomon's findings using the instruments 
developed by Salomon. Although he reports an acceptable degree of intemal consistency for the 
instrument used in his study, he states "... the validity of the AIME (Amount of Invested Mental 
Effort) instrument is a point of concern. Although self-reports about an intentional process 
like investing mental effort are possible in theory, validation studies in which mental effort is 
assessed by multiple methods are called for. " ( p. 56). 

Several researchers (Britton, Muth, & Glynn, 1986) have noted that the terms mental 
effort, attention, concentration, use of cognitive capacity, and mental workload all refer to 
similar concepts and refer to an increase in the cognitive resources devoted to processing the 
stimulus. In this paper, the relevant literature in communications, education, psychology, and 
human factors engineering will be reviewed to identify techniques that have been used 
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successfully to assess mental effort. The most promising methods will be described, studies 
which have applied these methods will be summarized, and the relationship between 
assessments of mental effort and achievement scores will be discussed. 

Potential Techniques and Methods 
Methods of assessing the construct of mental effort fall into three main categories: 
opinion measures, dual tasks techniques, and physiological measures. These categories parallel 
the areas of introspection, information processing, and neural processing identified by Posner 
(1982). 

Opinion measures 

Opinion measures encompass the variety of self-report measures used to assess mental 
effort. Opinion measures assume that the investment of effort is a voluntary process which is 
under the control of the individual, and as such, is available for introspection. 

Inventory of Learning Processes (IIP) scale The Inventory of Learning Processes (I LP) 
scale (Schmeck, R. R., Ribich, F. & Ramanaiah, N.,1977) consists of 121 true and false items 
designed to measure individual differences in learning processes. The test is divided into four 
subscales and cores on the Deep-processing and Elaboration subscales have been shown to have a 
significant correlation with scores on achievement measures (Gadzella, Ginter, & Williamson, 
1986; Gadzella, Ginter, & Williamson, 1987). The Deep-processing scale consists of 18 items 
which assess the extent to which the learners evaluate, analyze, and organize information. The 
Elaborative Processing scale consists of 14 items which assess how students translate and 
visualize information. 

Amount of Invested Mental Effort (AIME) questionnaire The A!ME questionnaire was 
developed by Salomon (1983) to assess learners' perceptic s of the amount of effort they invest 
in processing print and video-based instruction. Learners are asked to respond on a four point 
Likert scale of effort to a set of questions concerning the extent to which they concentrated 
during the lesson, how hard the lesson made them think, how hard the lesson was to understand, 
and similar questions. In a variety of studies (Beentjes, 1989, Cennamo, et, al, 1991; 
Salomon, 1983; Salomon, 1984; Salomon & Leigh, 1984; Salomon, Globerson, & Guterman, 
1989), as few as three and as many as eighteen questions have been included in the AIME 
questionnaire. 

Other opinion scales In addition to the scales used to assess the effort invested in 
educational tasks, several similar scales have been used to assess the mental workload involved 
in operating a flight simulator. The Cooper-Harper aircraft handling rating scale, Workload 
Compensation Interference/ Technical Effectiveness scale, and Multi-Descriptor scale (which 
required participants to rate the attentional demand, error level, difficulty, task complexity. 



ERLC 



4 



Assessing Mental Effort 
4 

mental workload and stress level of the task) have been used successfully in this performance 
tasks (Casali & Wierwille, 1983; Wierwille & Connor, 1983; Wierwille, Rahimi, & Casali, 
1985). 

Dual task techniques 

Dual task techniques encompass a range of methods which assign the learner a primary 
task such as reading a passage, working a problem, or viewing a videotape, and also assign the 
learner a secondary task such as responding to a tone, finger tapping, or estimating a time 
internal. Dual task techniques assume that there is a limit to the learner's cognitive capacity, 
and when a great deal of cognitive capacity is consumed by the primary task, there will be less 
capacity left to devote to the secondary task. Dual task techniques assume that the differences 
between performance on a baseline measurement of the secondary task and performance under 
experimental conditions is an indication of the amount of effort expended on the primary task. 

Finger tapp in g In the finger tapping method (Kee & Davis, 1988; Kee & Davis, 1990; 
Casali & Wierwille, 1983; Wierwille, Rahimi, & Casali, 1985), learners are directed to tap a 
key as quickly as possible. An electronic counter records the frequency of the finger taps. A 
baseline measurement is recorded prior to the introduction of the primary task. The decrease 
in performance on the finger tapping task is used as an indication of the effort expended in the 
primary task. 

Secondary task technique In the secondary task technique, (c. f. Britton, Muth & Glynn, 
19^"^; Gilbert & Scheuder, 1988; Meadowcroft & Reeves, 1985) learners are directed to press 
a key as quickly as pos$5ible following a visual or auditory signal. A baseline measurement is 
recorded prior to the introduction of the primary task. The increase in the time required to 
respond to the visual or auditory signal when attending to the primary task is assumed to be an 
indication of the effort expended in the primary task. 

Other dual task methods Other techniques that have been used in assessing mental 
workload when operating a flight simulator are digit shadowing, memory scanning, time 
estimation, and mental arithmetic tasks. In the digit shadowing technique, digits were presented 
on a screen and participants were required to read the digits aloud. A participant's score 
consisted of the number of digits presented but not read aloud (Wierwille & Connor, 1983). In 
the memory scanning task, participants heard a voice recording of digits and they were to push a 
lever to indicated whether the digits wore the same or different digits than those presented on a 
screen . The mean time required to correctly respond to the stimulus was used as an indication 
of mental workload (Wienwille & Connor, 1983). The time estimation task required the 
participant to press a lever ten seconds after receiving a cue to begin estimating the interval. 
The standard deviation of the participant's time interval estimates was used as an indication of 
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the effort invested in the primary task (Casaii & Wierwille, 1983; Wierwille & Connor, 
1983; Wierwille, Rahlmi, & Casaii, 1985). In the mental arithmet^s. task, participants were 
presented with verbal math problems requiring single digit responses. Participants responded 
verbally and their scores consisted of the percentage of incorrect responses to the problems 
presented (Wierwille & Connor, 1983). 
Physiological measures 

These techniques assume that there is a physiological response to increases in effort 
expenditure. The difference between a learners' baseline measurement of the physiological 
process and a measurement taken while the learner is performing some task is assumed to be 
reflective of the amount of effort the learner Is investing in the task. 

Heart rate In this technique, the pulse rate of the learner Is monitored to determine if 
there is an increase in heart rate while attending to the teaming task. It Is assumed that 
increased heart rate Is a natural physiological response to tasks that require information 
processing (Thorson & Lang, 1992). 

Electroencephalogram (EEG) measures The EGG records the electrical activity of the 
brain using electrodes attached to the scalp of the learner. It is assumed that the amount of 
alpha frequency present in the EEG varies inversely with the amount of mental activity 
(Reeves, Thorson, & Schleuder, 1986). 

Other physiological techniques Other measures that have been used in performance tasks 
include eye fixations, pupil diameter, number of eye blinks per minute, respiration rate in 
breaths-per- minute, and voice patterns as analyzed by a computerized psychological stress 
evaluator (Casaii & WienA^ille, 1983; Wierwille & Connor, 1983; Wienwille, Rahimi, & 
Casaii, 1985; Reeves, Thorson, Schleuder, 1985). 

Applications 

A series of studies in human factors engineering (Casaii & Wierwille, 1983; Wierwille 
& Connor, 1983; Wierwille, Rahimi, & Casaii, 1985) have attempted to determine the 
sensitivity of 15 different assessment measures for detecting variations in the workload placed 
upon an operator of a flight simulator (see Table 1). In these studies, the operators were 
presented with a series of tasks that ranged from easy to difficult to determine the effects of 
these manipulations on their scores on a variety of measures. The possible interference effects 
of the assessment measure on performance of the primary task was also investigated. The tasks 
performed by the operator were divided into perceptual tasks, psychomotor tasks, 
communication tasks, and mediational (cognitive problem solving) tasks. The researchers found 
that the techniques which were most sensitive to manipulations in the operator workload varied 
depending on the type of task performed by the operator. For example, eye blinks and eye 
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fixation measures were sensitive to increases in the difficulty of a cognitive task, but were not 
sensitive to increases in the difficulty of a communication task. Although one of the opinion 
scales (Modified Cooper-Harper) and the time estimation task were sensitive to increases in 
operator demands across all four types of tasks, the time estimation task resulted in an increase 
in the error rate of subjects who were performing the cognitive task. 



Insert Table 1 about here 



These results suggest that educational researchers who desire to assess the mental effort 
invested in learning tasks may need to take into consideration the type task required of the 
learner. The following section will describe a several studies which have investigated the 
amount of effort invested in a variety of perceptual and cognitive tasks. Although this review of 
the literature failed to locate instances where mental effort was assessed during the 
performance of communication or psychomotor tasks, the application of mental effort 
assessment techniques to tasks of this nature should not be neglected in future research. 

Perceptual tasks 

The following studies assessed the iiiental effort expended in various perceptual tasks. 
Dual Task techniques 

Secondary task technique. 

• The secondary task technique has been used to examine learners' responses to the 

perceptual demands of television processing. Reeves and Thorson (1986) found that 
cues that occurred curing a complex moment in a television commercial elicited longer 
reaction times than cues that occurred during less complex moments. Complexity 
involved both auditory elements such as the sentence syntax and visual elements such as 
pans, zooms, and movement within the scene. However, messages that contained a larger 
number of complex elements produced faster reaction times than messages that were less 
complex. Reeves and Thorson proposed that for simple messages, it may take more effort 
to construct meaning from the overall message than needed for more complex messages. 
The message in these experiments consibted of a series of 30-second commercials. 

• When learners were presented with a videotaped message that was presented over the 

audio channel, the visual channel, or both channels, their reaction times were slower 
when attending to both channels than when attending to either channel by itself. Audio- 
only required more effort to process than video-only, but this effect may have been due 
to the competition for auditory resources caused by the auditory probe (Reeves, 
Thorson, & Schleuder, 1985 ), 
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♦ The effects of competition for resources was evident in a study that investigated the effects 

of visual and auditory complexity using audio-only, video-only, or audio-video 
conditions. The researchers (Thorson, Reeves & Schleuder, 1985) found that the type 
of secondary task probe affected the pattern of learners' reaction times to a secondary 
task. When learners were presented with messages that were either visually simple, 
visually complex, auditorially simple or auditorilly complex under video-only, audio- 
only, or audio-video conditions, the channel that exhibited sensitivity to variations in 
complexity, as measured by slower reaction times to the secondary task, depended on the 
channel through which the secondary task probe was presented. When learners were 
asked to respond to a a'jditory tone, there were no significant main effects for auditory 
complexity and video simple messages produced longer reaction times than video complex 
messages. On the other hand, when learners were asked to respond to a light flash, there 
were no significant main effects for video complexity, but audio simple messages 
produced longer reaction times than audio complex messages. 
Reaction time measure 

♦ Gilbert and Schleuder (t988) had students look at black and white or color pictures and 

determine whether the caption matched the picture. The amount of time the students 
required to determine whether the caption and picture matched was assumed to be a 
reflection of the effort expended in the task. They found that students used less effort 
(faster response times) to correctly identify a match when viewing color photographs 
than with the black and white images. 
Physiological measures 

Electroencephalogram (EEG) measure 

♦ When Reeves and Thorson (1986) used a measure of EEG responses to determine 

learners' responses to the visual and auditory complexity of a series of 30-second 
television commercials, they found rapid declines in alpha (indicating increased 
attention) during complex moments in the commercial. These rapid declines in alpha 
could be attributed to the orienting response that occurs when an individual is presented 
with new stimuli. 

Cognitive tasks 

The following studies assessed the mental effort expended in performing various 
cognitive tasks. 
Opinion measures 
Self-reports 
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Bordeauz and Lange (1991) asked children in grades two, four, and six about the effort 
they invested in nine types of television programs (cartoons, children's programs, 
family shows, situation comedies, news or documentary, entertainment shows, adventure 
or action programs, sports, and soap operas). These investigations examined learners 
perceptions of their overall effort investment in a particular type of program, and as 
such, are reflective of learners preconceptions of the effort required to process 
particular program types. However, these investigations differ from those that present 
the learners with a specific television program and ask the learners to rate the amount of 
effort invested in that specific program. 

Salomon (1983; 1984; Salomon & Leigh, 1984) used self-report questionnaires to 
assess the amount of mental effort that learners perceived they invested in processing 
television and print. In these studies, sixth-grade students were presented with a story 
to read or to view on videotape. They consistently found that the students reported 
investing less effort in processing the video-taped version than in processing the text- 
based version of the story. In general, they reported that students who read the print- 
based version performed better on a test of recall and inference items than the students 
who viewed the videotaped version. In addition, students who were instructed to learn 
from the story reported investing more effort in processing the story than those who 
were instructed to read or view the story for fun. 

Cennamo, Savenye and Smith (1991) used a similar questionnaire to assess learners 
perceptions of the amount of effort they invested in processing interactive video and 
video-based instruction. Learners either viewed a videotape of a science lesson, viewed a 
videotape of the same lesson with practice questions and feedback, or completed a 
videodisc version of the lesson that required the learners to actively respond to the 
embedded questions before they would receive feedback on their response. The 
researchers found no significant difference among the groups in their ratings of invested 
effort. 

Chu (1987) used a self-report questionnaire modeled after Salomon's in her study that 
used computer-assisted instruction to examine the effects of student generated 
underlining, program generated underlining, and free choice of the underlining option on 
learners' perceived mental effort and achievement. There were no significant differences 
in performance among the three groups on a recall posttest, and, unfortunately, she did 
not compare the mental effort ratings among groups. However, she did report a weak but 
significant correlation between learners perceptions of the task difficulty and their 
perceived mental effort ratings. 
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Dual task techniques 
Finger tapping 

• Kee and Davis had students tap while they attempted to learn various noun pairs that were 
presented aurally. Students were asked to use a variety of strategies as they memorized 
the noun pairs: repeating silently or orally (1988), generate a sentence with the noun 
pairs orally (1988) or silently (1988, 1990), learning a sentence containing the noun 
pair (1990) or however they wished (1990). Although the strategies used for 
memorization did not affect finger tapping speed significantly in the majority of cases, 
finger tapping speed decreased significantly when the learners were memorizing noun 
pairs with no logical connection using the strategy that required them to generate a 
sentence containing both nouns. 

Secondary task technique 

The secondary task technique has been used extensively in a variety of research studies that 
examined students' performance of cognitive tasks. 

• Meadowcroft and Reeves Qused the secondary task technique to determine the amount of 
effort that children invested in processing cartoons. The cartoons were presented in a 
jumbled or normal form. The researchers found that the students allocated more 
attention to scenes that were central to the story content than to extraneous scenes. 
Children that appeared to have well developed schema exerted less effort in processing the 
content of the programs than those who had low levels of schema development. 

• Grimes (1990) used a secondary task technique to determine the effects of the degree of 
match between the audio and video portions of a television program. He presented the 
learners with a program that either had a high degree of correspondence between what 
was talked about and seen on the screen, a thematic match between the two channels 
(medium correspondence) or no correspondence between the two channels. The learners 
were told to learn all that they could for a test following the program. Grimes presented 
the secondary task over either the visual or auditory channels. He found no significant 
differences among reaction times to the auditory probe among the three correspondence 
conditions; however, when a visual probe was used, the high correspondence condition 
resulted in significantly slower reaction times than the other conditions. 

• Cameron, Schleuder, and Thorson (1991) had learners watch a series of newscasts that 
included commercial breaks. Half of the commercials breaks were preceded by teasers 
and half were not. The learners were told that they would be tested over what they iiad 
seen. A secondary task technique was used to determine the effect of news teasers on the 
learners attention to the commercials. The reaction times to tones which occurred during 
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commercials preceded by teasers were significantly slower than the reaction times to 
tones which occurred during commercials with no teasers. With teasers, the middle 
commercial elicited the longest reaction times, but when there were no teasers, the first 
and last commercial elicited longer reaction times. 

• Shapiro (1985) had participants respond to a visual secondary task (a light flash) while 

« 

listening to an audio-taped science passage. He examined the effects of the presence of 
analogies and the effects of visualization training on learners reaction times and 
achievement scores. His results suggest that learners who received the lessons with 
analogies and learners who were instructed to visualize the content may have used more 
effort during the processing of the lesson; however, there were no significant differences 
on recall scores. 

• Reed, Burtor* and Kelly (1985) used a secondary task technique to determine learners 
cognitive engagement during a writing assignment. English students enrolled in an honors 
class, an average class, and a remedial class were asked to write a narrative, persuasive, 
or descriptive essay. They found that the writers ability differentially affected their 
cognitive engagement across modes of writing. For example, honors writers had the 
longest reaction times to the secondary probe when writing persuasive essays, but 
shorter reaction times when writing descriptive essays. Average students, on the other 
hand, exhibited their longest reaction times when writing descriptive essays. 

• Britton and other researchers have used the secondary task techniques extensively in 
their investigation of the processing of text-based materials. They have presented 
learners with a variety of printed messages to determine the effects of prior knowledge of 
the content (Britton & Tesser, 1982), meaningfulness of the passage (Britton, 
Holdredge, Curry & Westbrook, 1979), expectations of delayed and immediate tests 
(Britton, 1980), narrative and expository text (Britton, Graesser, Glynn, Hamilton, & 
Penland, 1983), embedded questions(Britton, Piha, Davis, & Wehausen, 1978; Burton, 
Niles, & Lalik, 1986), interest (Shirey & Reynolds, 1988), text syntax (Britton, 
Glynn, Mayer, & Penland, 1982), objectives (Britton, Glynn, Muth, & Penland, 1985). 

Letter identification task 

• White (1986) used a letter identification task to measure the workload on the sensory 
register. Subjects viewed videotaped PSAs (public service announcements) of varying 
complexity levels and were asked to simultaneously pull a lever when they identified 
letters which also appeared on the screen. As the complexity of the television image 
increased, performance on the letter identification task decreased. The author concluded 
that the letter identification task may have interfered with learning. 
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Physiological measures 
Heart rate 

• Although learners' heart rates were monitored to determine if there were differences in 
the effort expended in processing vi^isotaped lectures covering familiar and unfamiliar 
content and including complex and simple video graphics, the heart rate patterns did not 
show any significant differences in the effort expended in processing the four types of 
lectures (Thorson & Lang, 1992) 

Correlation of Mental Effort Measurements and Achievement 

Opinion measures 

Self-report questionnaires 

Salomon found a significant correlation between learners self-reports of invested 
mental effort and their scores on a test of inference items. In general, he found that learners 
who received a print-based version of a lesson reported investing more mental effort in 
processing the lesson than those students who received a video-based version of the lesson, in 
addition, the students who received the print-based version achieved higher scores on a test of 
inference items than those who received the video-based version. Cennamo, Savenye and Smith 
(1991) also reported a weak, but significant correlation between learners self-reports of 
invested mental effort and their scores on a test of recall and inference items. They found that 
learners' test scores tended to increase as their perceived mental effort ratings increased. 
Inventoi ' of Learning Processes 

This instrument is administered prior to a learning task and assesses learners 
tendencies to elaborate and process deeply. Learners who scored high on the deep processing 
scale had higher scores on a test of free and cued recall following a cooperative learning activity 
where they had to read and study a passage describing a piece of technical equipment, stopping 
periodically to summarize the information to one another, than learners who with low scores on 
the deep processing subscale (Skaggs, et. al., ). Other researchers (Gadzella, et. al., 1986) 
found that there was a significant correlation between students overall grade point average and 
there scores on deep processing subscale. They also found that students who completed a 
computer-assisted instructional lesson designed to teach study skiPs showed a significant 
correlation between scores on the deep processing subscale and their scores on a test assessing 
their knowledge of some study skills (Gadzella, et. al. 1987). 
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Dual task techniques 

Finger tapping 

Kee and Davis found significant differences for recall in several instances where there 
were no significant differences in speed of finger tapping. Students who were asked to generate a 
sentence recalled significantly more noun pairs than students who rehearsed the pairs (1988; 
1990) and that logical pairs were more easily recalled than illogical pairs (1990); however, 
finger tapping speed was not significantly altered except for the instar ce where learners were 
required to generate a sentence containing a pair of nouns that had no ivjgical connection. 
Secondary task technique 

There are four possible patterns of scores on achievement measures and a reaction times 
to a secondary task: a) longer reaction times may correspond to higher recall scores, b) longer 
reaction times may correspond to lower recall scores, c) there may be no significant difference 
in reaction times among the groups, yet there may be significant differences on recall 
measures, or d) there may be significant differences on reaction times and no significant 
differences in recall scores. 

Longer reaction times- Higher recall scores . 

• Grimes (1990) found the same pattern of scores on a visual recognition test and reaction 
times to a visual probe. He found that reaction times were slowest for the high 
correspondence condition, and fastest for the medium correspondence condition. Scores 
on the visual recognition test followed the same pattern, with learners who viewed the 
program with high correspondence scoring highest, followed by those who viewed the 
program with r>o correspondence, and finally, t!iose who viewed the program with only a 
thematic match between the visual and the audio portion of the program. 

• In a task that required learners to write a variety of essays. Reed, Burton and Kelly 
(1985) found that reaction times to a secondary task were positively related to scores of 
writing quality. 

• Britton, Muth and Glynn presented learners with a text passage and found that when there 
were no time pressures, students spent more time in the reading of important 
information than in reading less important information; however, when the learners 
were restricted in the time spent on reading the text passage, more effort was spent on 
processing important sentences than unimportant ones. Important information was also 
recalled more often than unimportant information. 

• Briton, Westbrook and Holdredge (1978) found that learners exerted more effort in 
reading easy passages than in reading difficult passages and they also recalled 
significantly more information from the easy passages. 
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• Britton, Glynn, Muth, and Penland found that learners who were aware of the specific 
objectives of a passage exerted more effort in processing the passage and also recalled 
more information from the passage than learners who were presented with a general 
objective or no objectives. 

Longer reaction times- Lower recall scores 

• Cameron, Schleuder, and Thorson (1991) found an interesting relationship between 
learners' scores on a multiple choice visual and verbal recognition test and iheir 
reaction times to a secondary task. In an experiment that examined the effects of news 
teasers on learners attention to commercials during a newscast, they found that verbal 
memory was higher for the first and last commercials during a commercial break that 
had been preceded by a news teaser, but that reaction times for the first and last 
commercial were shorter (so presumably less effort was expended) than for the middle 
commercial. 

• Shirey and Reynolds (1988) presented learners with a series of sentences on a computer 
screen. They found that students recalled sentences rated as interesting at a higher rate 
than those rated less interesting; however, less effort was allocated to the interesting 
sentences than the uninteresting ones. More effort was allocated to reading uninteresting 
sentences. 

• Britton, Glynn, Mere, and Penland found that learners who read a technical text that was 
written using common words answered more questions correctly than learners who read 
the technical text that used rare words. In addition, those students who read the 
technical text written with simple syntax answered more multiple choice questions 
correctly than those who read a version written with complex syntax. The reaction time 
measures indicate that learners allocated more effort to reading the text with complex 
syntax than to reading the text with simple syntax. The differences in reaction time 
between those who read the text with common and rare words were not significant. 

• Lang (1991) found that learners exerted more effort in processing video segments that 
included video cuts that were represented changes in content than they did in processing 
cuts that were related in content to the preceding information. However, the learners 
recalled more of the information surrounding related cuts than surrounding unrelated 
cuts. 

Difference in reaction times- No difference in recall scores 

• Reeves, ThorsOn, and Schleuder (1985) found no significant differences in learners 
performance among the audio-only, video-only or audio-video conditions. However, 
there were significant differences among the three groups in reaction times to a tone; 
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reaction times were significantly delayed for those who attended to the audio-video 
presentation. 

♦ Often there are significant effects on performance measures with no corresponding 
effects on reaction times to a secondary task; however, in a study that examined the 
effects of analogies and visualization training on learning a science passage, Shapiro 
(1985) found significant effects on reaction times with no corresponding effects on an 
achievement measure. 

♦ Another experiment (Britton, Glynn, Mere, & Penland, 1982) found that learners who 
read a text that included text "signals" (such as therefore, consequently, end likewise ) 
allocated less effort to reading the text than learners who read a version that did not 
include signals. However, there were no significant differences among the groups on a 
measure of free recall. 

♦ Britton (1980) found that learners who expected a delayed test used more effort in 

reading the passage than those who expected an immediate test; however, there were no 
significant differences between the groups in the amount of information recalled. 

♦ Although Britton, Holdredge, Curry and Westbrook (1979) found that learners exerted 
more effort in reading passages which contained a title that related the sentences to each 
other than in reading the same passage that was meaningless without the title, there was 
no significant difference between the two conditions on recall scores. 

No difference in reaction times- Difference in recall scores 

♦ Readers recalled more information from a paragraph that appeared in a context that 
emphasized the importance of the content than they did from the same paragraph in a 
context that minimized its importance; however, there was no significant difference in 
the learners' reaction times between the two conditions ( Britton, Mere, Simpson, 
Holdredge, & Curry, 1979). 

Physiological measures 
Electroencephalogram (EEG) measure 

Reeves and Thorson (1986) found that lower mean levels of alpha in response to a 30- 
second television commercial correlated with increased memory of the content of the message. 
Heart rate . 

Thorson and Lang (1992) examined the effects of video graphics inserted into videotaped 
lectures on content with which the learners were either familiar cr unfamiliar. Although they 
found significant differences among the treatment conditions on a measure of recall, there were 
no significant differences among treatments on the heart rate patterns. Memory for information 
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following the video graphic in the familiar lectures was improved, but memory for information 
fo'' ^wing the graphic in the unfamiliar lectures was decreased. 

Conclusions 

Winn (1986) indicated that an investigation of ways that designers can alter the 
perceptions of learners to increase the amount of mental effort invested in processing the lesson 
may be an especially promising area for further research. However, the search for techniques 
to Increase the mental effort invested in processing a video-based lesson has been hindered by 
the methodology available to assess the construct of mental effort. There is a need for 
educational researchers to determine the sensitivity of techniques of assessing mental effort to 
increases in the processing requirements of a losson; this literature review is an initial step in 
that direction. 
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Table 1 

Effectiveness of techniques used to assess mental workload 



Assessment Perceptual Psychomotor Communication Mediational 

Measure (cognitive) 

OPINION MEASURES 

Modified Cooper Harper scale X - XX 

(Cooper Harper scale in psychomotor) X 

Multi-descriptor scale X - X O 

Workload compensation interference/ XX - X 
technical effectiveness scale 

SECONDARYTASKS 

Time estimation standard deviation XX XX 

Tapping regularity X - O O 

lOigit shadowing - O - - 

Memory scanning O 

Mental arithmetic - 0 - - 
PHYSIOLOGICAL 

iiespiration rate X 0 O O 

Heart rate mean , O X O O 

Heart rate variability (SD) 0 O 0 O 

Pupil diameter O 0 X O 

Eye blinks O - OX 

Eye fixations - - OX 

Voice pattern scores . O - - 



KEY 

0 not sensitive to differences in workload 
X sensitive to differences in workload 
not assessed 
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